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ALGORITHMIC  EQUIVALENCE  IN  QUADRATIC  PROGRAMMING  I: 

A LEAST-DISTANCE  PROGRAMMING  PROBLEM 

Richard  W.  Cottle  and  Arthur  Djang 
ABSTRACT 

It  is  detaonstrated  that  Wolfe's  algorithm  for  finding  the  point 
of  smallest  Euclidean  norm  in  a given  convex  polytope  generates  the  same 
sequence  of  feasible  points  as  does  the  van  de  Panne -Whinston  "symmetric" 
algorithm  applied  to  the  associated  quadratic  programming  problem.  Further- 
more, it  is  shown  how  the  latter  algorithm  may  be  simplified  for  application 
to  problems  of  this  type. 


1.  Introducti on 

This  is  the  first  of  a series  of  pap'^rs  wMch  are  intended  to 
provide  a conceptual  basis  for  the  analysis  and  coHipai-ison  of  quadratic 
programming  algorithms.  The  series  considers  a wide  variety  of  quadratic 
programming  problems j ranging  from  those  having  certain  special  structure 
isuch  as  the  one  discussed  in  this  paper)  to  more  general  ^e.g.  r.onoonvex < 
problems.  With  regard  to  these  problems,  the  concept  cf  equivalence  between 
algorithms  (in  the  sense  of  generating  identical  solution  paths'  is 
examined.  Apart  from  their  theoretical  interest,  nhe  results  developed 
in  this  series  of  papers  should  be  of  interest  to  those  concerned  with 
finding  computationally  efficient  methods  for  solving  quadratic  programming 
problems. 

The  beast-Distance  Problem  (LDP) 

In  Wolfe  introduces  an  algorithm  for  finding  that  point  of  a 

given  polytope  having  smallest  Euclidean  norm.  The  problem  under  con- 
sideration differs  from,  least-distance  problems  studied  by  certain  other 
authors  'e.g.  Tucker  [7]’',  in  that  the  given  polytope  is  described  as 
the  convex  hull  of  a given  point  set,  rather  than  as  the  intersection 
of  iuilfspaces . 

In  particular,  let  F = (P.^^ , , . . . , P^. } be  a given  set  of  distinct 

nonzero  points  in  e'"‘.  The  least-distance  problem  (LDP)  can  be  stated  as; 
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minimize 


r 

k 

I 


i 

||x||"  - X^x 

m 

subject  to  X - P,  w, 
k=l 
m 

Z w,  =1,  all  w,  > 0 . 
k=l 

As  Wolfe  [lO]  points  out,  problems  of  this  type  arise  in  applica- 
tions such  as  the  minimization  of  nondi fferentiable  functions  and  pattern 
recognition.  For  problems  in  which  m < n,  Wolfe  [11,12]  recommends  the 
use  of  the  "tableau  variant"  of  his  LDP  algorithm.  For  problems  with 
m » n,  he  suggests  the  use  of  other  variants  of  the  method;  they  follow 
the  same  path  geometrically  but  organize  the  calculations  in  a more 
efficient  manner. 

Outline  of  the  paper 

Section  2 of  the  paper  provides  a brief  description  of  Wolfe's 
LDP  algorithm;  it  also  discusses  the  "symmetric"  quadratic  programming 
algorithm  of  van  de  Panne  and  Whinston  [6]  as  specialized  to  the  LDP. 

For  brevity,  the  latter  method  will  be  referred  to  as  the  "S-algorithm. " 

Section  3 shows  that  the  structure  of  the  LDP  permits  considerable 
simplification  of  the  S-algorithm.  It  also  examines  the  special  structure 
of  the  tableaux  generated  by  this  algorithm  and  demonstrates  that  the  LDP 
may  be  solved  using  principal  pivots  exclusively.  Finally,  the  section 
includes  a geometrical  interpretation  of  the  tableaux  elements  which 
is  based  on  Wolfe's  [ll]  analysis  of  the  "tableau  variant"  of  his  LDP 
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algorithm. 


Section  H demonstratea  that  l.he  S-aigor ! Uim  ar;:!  Wi>lfe'a  algorithm 
generate  the  same  sequence  of  feasible  points  when  applii-d  to  tfie  IJJI . 
Fiirthermore,  the  S-algoritiun  and  Wolfe's  "tableau  variant"  generate  the 
same  sequence  of  tableaux,  provided  that  the  inte riue di ate  tableaux 
produced  during  the  so-called  minor  cycles  of  each  method  are  eliminated 
from  consideration.  Since  the  two  methods  perforin  an  equivalent  amourit 
of  work  in  each  minor  cycle,  we  conclude  that  the  S-algori*hm  arid  the 
"tableau  variant"  of  V/olfe's  I, DP  algorithm  are  equivalent  in  terms  of  the 
computational  effort  required  to  solve  the  LDP. 

The  appendix  of  the  paper  is  devoted  to  a discussion  of  degeneracy 
in  the  LDP,  and  to  a summaiy  of  von  Hohenbalken' s method  [^'\  for  maximizing 
pseudoconcave  functions  on  polytopes.  When  applied  to  the  LDP,  this  method 
is  identical  to  Wolfe's  algorithm., 

■ ^r-i'pf  Summary  of  the  Algorithm.s 

a.  Wolfe's  LDP  algorithm.  Initially,  VJolfe  fll]  expresses  his 
i,;orithms  for  the  I,Di’  in  geometric  language.  i.ater,  lie  puts  the  problems 
in  a tabular  format  and  identifes  the  geometric  significance  of  the 
tableau  entries.  For  the  convenience  of  the  reader,  we  review  the 
notations  'ind  terminology  used  in  fll]. 

Let  Q denote  a set  of  k.  column  vectors  ipoints)  in  Euclidean 
r.-.-pa'ie,  E^.  It  will  facilitate  our  discussion  to  assemble  this  set  of 
vectors  as  an  n w k ma.trix,  also  denoted  Q.  The  affine  hull  of  y. 
is  the  set 
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A(Q,)  = {X  G E^;X  = -Qw,  e\-  = il 


where  ■=  = (w, , . . . ,w,  ) G E^.  The  convex  hull  of  Q, 

i-  K 

is  the  set 

C(q)  = {X  G e”;X  = Qvr,  e^^w  = 1,  w > 0}  . 

The  set  Q is  affinely  independent  if  q G A(q\  (q])  Is  false  for  all 
q G Q.  (Note  that  the  backward  slash  represents  set-theoretic  difference 
and  (q)  is  the  set  with  one  element:  q.  ) A point  X G ' \ (0] 
determines  a hyperplane 

h(x)  = (Y  G E’^rY^X  = X^X) 

passing  through  X and  normal  to  the  line  OX.  The  notation  o/h/S 
means  that  the  hyperplane  H separates  the  set  S from  the  origin  0, 
while  the  notation  O/S/h  means  that  the  set  S lies  on  the  near  side 
of  the  hyperplane  H.  (Wolfe  does  not  use  these  notations,  but  he  does 
use  the  concepts  they  represent. ) An  affinely  independent  subset  Q 
of  P (the  given  set  of  k points)  is  a corral  if  the  point  of  least 
norm  in  C(q)  belongs  to  the  relative  interior  of  C(q).  By  convention, 
a singleton  is  a corral. 

Wolfe' s algorithm  consists  of  a finite  number  of  major  cycles , 
each  of  wiiich  consists  of  a finite  number  of  minor  cycles . In  the  first 
two  flow  charts  below  the  major  cycles  begin  at  step  (l);  the  minor 


k 


cycles  consist  of  steps  (2)  g.nd  (3).  At  the  beginning  of  each  major  cycle, 
a corral  Q.  and  the  point  X € C(Q)  of  least  norm  are  known.  It  should 
be  noted  that  the  solution  to  the  problem  has  this  form  for  some  corral  Q, 
The  proof  that  the  algorithm  temlnates  in  a solution  after 
finitely  many  steps  is  based  upon  the  following  facts: 

1.  Q is  always  affinely  independent;  it  char;ges  only  by  the  deletion  of 
single  points  or  by  the  adjunction  of  P.  € P in  step  (l). 

2.  The  number  of  minor  cycles  (if  any)  with  a single  major  cycle  is  at 
most  the  dimension  of  C(q). 

T 

.?•  The  value  of  the  objective  function  XX  is  reduced  in  each  major 
cycle. 

i . No  corral  can  enter  the  algorith.m  more  than  once  as  the  point  X is 

T 

uniquely  determined  by  the  corra]  (and  by  the  previous  fact,  X X 
ci'.'creases  from  one  ma.jor  cycle  to  the  next). 

The  tableau  variant  of  Wolfe's  algorithm  (see  Flowchart  ?)  has 
■'he  following  properties: 

U In  the  step  imiriediately  following  the  optimality  test,  the  point.  P. 

u 

T 

■'or  adjunction  to  Q Is  that  which  mlniroizes  X P.  from  among  those 
points  It  for  which  o/P^/H(X). 

(b)  The  step  which  finds  the  point  of  minimum  nom  in  AvqI  (see  step  ^2)) 
is  accomplished  by  a principal  pivot. 
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We  apply  the  S-algorithm  to  the  following  convex  quadratic 
programming  problem,  which  is  equivalent  to  the  LDP; 

ITT 

minimize  — w P Pw 
T 

subject  to  e w = 1 
w > 0 . 

The  Kuhn-Tucker  conditions  in  tableau  form  are; 


r.T 

P P 

-e 

0 

W-r 

0 

-1 

J 

T 

with  ^ '^J  “ Associated  with 

the  above  initial  tableau  are  index  sets  I = {l,2,...,m]  and  J = {m+1], 

I is  the  set  of  indices  of  the  basic  dual  variables,  and  J is  the  set 

of  indices  of  the  basic  primal  variables.  As  the  algorithm  progresses, 

the  sets  I and  J will  change. 

The  S-algorithm  generates  a sequence  of  adjacent  basic  solutions 

of  the  Kuhn-Tucker  equations  via  pivot  operations,  each  denoted  (B.V.,D.V.). 

The  D.V.  ("driving  variable")  is  a nonbasic  variable  whose  value  is  being 

* 

increased;  it  is  chosen  to  be  the  complement  of  the  most  negative  basic 

Note:  The  variable  w^  is  said  to  be  the  complement  of  the  variable  y^. 

In  this  paper,  a tableau  will  be  termed  "complementary"  if  = 0 

for  all  i,  1 < i < m+1.  Otherwise,  if  > 0 for  some  i,  the  tableau 

is  said  to  be  "non-complementary." 
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dual  variable  (the  "disuirifmished  variable"). 


t 


The  B.V.  ("blocking  variable")  is  a basic  variable  v/hose  value 
either  decreases  to  zero  (if  the  B.V.  happens  lo  be  one  of  the  basic 
primal  variables)  or  increases  to  zero  (if  the  B.V.  happens  t,o  be  the 
distinguished  variable)  as  a result  of  the  increase  in  the  D.V.,  and  hence 
leaves  the  basis.  (Basic  dual  variables  other  than  the  distinguished 
variable  are  not  eligible  to  play  the  role  of  B.V. ) The  pivot  operation 
consists  of  solving  the  equations  for  the  new  set  of  basic  variables. 

Starting  from  the  initial  tableau,  the  S-algoritlim  performs  an 
"exchange-pivot"  or  2 k o principal  block  pivot.  This  yields  a primal 
feasible  point  wldch  is  a member  of  the  set  P = (P^,...,P^]  having 
minimum  norm. 

Denote  a typical  tableau  T generated  by  the  S-algorithm  (after 
'he  initialization  step)  as; 


1 

— 

JI 

'll 

«.!  J 

:,'o^c  that  here  the  set  J is  not  necessarily  a singleton  (as  it  was  in 
initial  tableau). 

Three  types  of  pivot  operations  can  be  performed,  each  of  which 
will  ensure  that  each  successive  tableau  generated  by  the  method  possesses 
he  following  properties: 
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(l)  Bisyranetry;  After  the  pivot,  the  new  tableau  is  of  the  following 
form,  where  and  Mj,j,  are  symmetric; 


(2)  Positive  semidefiniteness  of  the  matrix  M , (and  also  of  the 

J tJ 

matrix  in  case  of  convex  quadratic  programming). 

(3)  Primal  feasibility:  qj,  > 0 . 

The  S-algorithm  terminates  when,  in  addition  to  these  properties,  the 
tableau  is  also  dual  feasible:  y^,  = q^,  > 0. 

The  following  types  of  pivot  operations  may  be  performed; 

(a)  IW-PIVOT  where  t € I. 

I This  1x1  principal  pivot  increases  #J,  the  number  of  basic  primal 

I' 

variables,  by  one. 

(b)  OUT-PIVOT  (Wg,yg)  where  s € J. 

This  1x1  principal  pivot  decreases  jfj  by  one. 

(c)  EXOUANOE-PIVOT  (w  ,w.  ^ t G T,  s € J. 

In  this  2^2  block  principal  pivot,  the  number  of  basic  primal 
variables  remains  the  same.  The  operation  is  performed  by  making 
two  1x1  non-principal  pivots.  As  will  be  shown,  this  type  of  pivot 
is  not  required  when  the  algorithm  is  applied  to  the  LDP.  i,Tn  fact, 
even  the  initialization  step,  which  is  nominally  accomplished  by  an 
exchange-pivot,  may  be  carried  out  using  1x1  principal  pivots  only.) 
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Convergence  of  the  S-algoritlim  for  convex  quadratic  progrfumning 
problems  Is  established  in  the  following  manner  r/an  de  Panne  and 
V.Tfiinston  [8]). 

If  some  basic  dual  variable  y (t  £ I ) in  the  current  comple- 
mentary  tableau  is  negative,  the  tableau  does  not  represent  a minimum. 

t The  complement  of  y , namely  w, , is  a nonbasic  primal  variable  whose 

r ■ 1.  X 

I 

value  is  then  increased.  One  of  the  three  types  of  pivot  operations  will 
be  performed. 

If  the  pivot  is  of  type  (a)  or  (c),  we  get  a new  primal  feasible 

T T T 

complem.entary  solution,  for  which  f(w)  = w P Pw  = X X has  reduced  value. 

However,  if  the  pivot  is  of  type  (b)  (,an  out-pivot  where  s £ j), 

the  driving  variable  w will  be  nonbasic  in  the  new  tableau,  but  will 
ha"e  positive  value--the  value  at  which  its  increase  was  blocked  by  w^ 
decreasing  to  zero.  Thus  the  new  tableau  is  "non-complementary"  in  the 
sense  that  the  variables  w^  and  y^  both  have  nonzero  values.  However, 
rngr-e  n-xxi  cn ly  he  m such  pivots  of  type  ',b)  i.n  succession,  since  each  ; 

step  reduces  t.he  cardinality  of  .J  (denoted  by  fij)  by  one,  and 
! :'i  < II-]  ^ m.  Eventually,  the  algorithm  must  generate  a new  complementary 

solution,  thus  completing  a "major  cycle";  the  value  of  f(w)  is  then 
strictly  lower  than  its  value  at  the  beginning  of  the  iriajcr  cycle. 

Assuming  that  each  basic  solution  is  nondegenerate,  we  can  assert 
that  since  there  exist  a finite  number  of  complementary  bases,  and  since 
in  each  major  cycle  the  value  of  f(w)  is  reduced,  no  basis  can  repeat-- 
hence  the  algorithm  will  terminate  with  the  solution  in  a finite  number 
of  steps.  (Appendix  I of  this  paper  demonstrates  that  the  assumption  of 
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nondepeneracy  can  actually  be  dropped  for  the  purpose  of  proving  >.hat 
the  S-algorithm  will  converge  to  the  solution  of  the  LDP. 


c.  Flow  charts  of  the  algorithms.  To  facilitate  future  reference 
and  discussion  we  have  represented  Wolfe's  LDP  algorithai  and  the  van  de 
Panne-Wiiinston  S-algorithm  in  flowchart  form.  Violfe's  algorithm  has  been 
given  in  both  geometric  and  algebraic  (.tableau)  forms. 

rt  is  hoped  that  the  reader  will  notice  the  flowcharts  have  pre- 
cisely the  same  structure.  This  property  warrants  a comr.ent.  The 
vn.n  de  Panne-Whinston  symmetric  algorithm  for  convex  quadratic  programming 
involves  the  use  of  1 x 1 diagonal  pivots  and  2x2  block  diagonal  pivots. 
One  of  the  contributions  of  this  paper  is  a demonstration  that  only 
diagonal  pivots  are  necessary  wFien  the  van  de  Panne-FJhinston  S-algorithm 
is  specialized  to  the  LDP  problem.  This  fact  is  already  incorporated 
in  the  corresponding  flowchart  and  is  indicated  there  by  the  comment  in 
the  lower  right-hand  corner  of  the  page.  To  a great  extent,  it  accounts 
for  the  duplication  of  the  flowchart  structures. 
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Let  X be  that  point  an<i  Q = {XI 


If X = 0 or 0/HiX'  'V, stop;  X is  optimal 


the  face  of  C^Q)  in  which  Z lie 


GEOMETRIC  FORM  OF  WOLFE 'G  LbP  ALGORITHM 


INTTIALIZJ\TION 


0 Find  a point  P.  P of  minimal  norm. 


START  MAJOR  CYCLE  v/ith  a corral  | Q and  a p^oint  X c C(Q)  of 
minimal  norm 


OPTIMALITY  TEST 


Otherwise  choose  P^.  suchi  that  0/p^/H(X)  and  adjoin  P^  to  Q 


Find  Y,  the  point  of  smallest  norm  in  A(Q'. 


This  IS  done  by  solving  the  system 


and  setting  Y = Qy. 


FEASIBILITY 

TEST 


Replace  X 


j(3^|  Find  Z , the  point  nearest  to  Y on  the 
! ; line  segment  C(Q)  0 XY. 


I Delete  from  Q one  of  the  points  not  on 


TABLEAU  VARIANT  OF  WOLFE'S  LDP  ALGORITHI-1 


Start  with  the  (m+1)  x (tn+1)  tableau  T^  = 
Number  the  rows  and  columns  {0,1,..., m). 


INITIALIZATION 


VAN  DE  PANNE-WHINi:TON  S -ALGORITHM  SPECIALIZED  TO  THE  LDP 


Itart  with  the  tableau 


Denote  tlie  constant 


column  of  q,  and  the 
rest  of  the  tableau 


INITIALIZATION 


Let  i denote  the  index  of  the  smallest  element  of  the  main  diagonal 


EXCHANGE-PIVOT 


V . > 0,  i J,  and  w.  =0,  i G I 
the  current  feasible  point  X 


START  MAJOR  CYCLE.  The  variables 
are  the  barycentric  coordinates  of 
with  resoect  to  P. 


OPTIMALITY  TEST 


> 0,  stop:  X = Pw  is  optimal 


min 

iGI 


Otherwise,  increase  the  driving  variable  w. 


/m  . = max  { q ./m . , :m . , < 0] 
s'  st  jt  jt 


Ratio  test:  Find  s G J such  that 


If  m,  , > 0,  compare  the  values  of  q /m  ^ and  q. 


m, , > 0;  see  Theorem  3 


the  LDP,  we  always  hav 


nj-prvDT 

Transfer  the  index 


FEASIBILITY 

TEST 


from  I to  J 


blocks  the  increase  of  w 


0,  perform  the 


the  exchange-pivot 


OUT -PIVOT  ( 


must  be  performed. 
However,  for  the  LDP 
we  will  always  have 

m >0  (see 
ss 

Theorem  d. 6 ' . 


Transfer  the  index  s from  J to  I. 
Continue  the  increase  of  the  driving 
variable  w,  . 


T 

P P 

-e 

0 

T 

e 

0 

-1 

d 


Example  of  an  I.DP  Problem  (due  to  Wolfe  [ 11 J ) 


Problem: 


minimize  X X 


subject  to  X ^ E P,  w , E w = 1,  w > 0 
k=l  ^ k=l  ^ ^ 


where  P = [ P^  P^ . 


' 0 3 -2" 

"201' 


The  S-algorithm,  applied  to  this  problem,  generates  the  following  sequence 
of  tableaux: 


^1 

^2 

"3 

1 

^1 

h 

0 

2 

-1 

0 

Y2 

0 

-6 

-1 

0 

^3 

2 

-6 

5 

-1 

0 

1 

1 

1 

0 

-1 

The  initial  point  having  barycentric 

coordinates  w = (0,0,0)  with  respect 

to  P is  not  feasible  since 
T 

e V f 1.  Apply  initialization  step. 


INITIALIZE:  EXCHANGE -PIVOT  (w^,w^\,  (y^,yj^) 


^^2 

^3 

^1 

1 

^2 

13 

-k 

-k 

1 

-k 

^3 

-k 

5 

-2 

1 

-2 

-k 

-2 

k 

-1 

k 

W 

1 

-1 

-1 

1 

0 

1 

Tableau  corresponds  to  the  primal 
feasible  vertex  P^  having  bary- 
centric coordinates  w = (l,0,0). 


Increase  w^,  since  y^  is 


distinguished. 


IN-PIVOT 


1 


w.. 


\ 


OUT- PIVOT  (w  , 


I IN-PIVOT  {y~^ 


'"1 

'"U 

Y2 

1 

^1 

7 

17 

1 

y-^ 

17 

26 

-15 

1 

-15 

-9 

-15 

9 

-1 

9 

'"2 

-1 

-1 

1 

0 

1 

1 

^1 

>2 

1 

^1 

ii-9 

21 

17 

21 

1 

21 

9 

-11 

-15 

9 

'■’2 

26 

-9 

11 

1 

_ 1 

11 

-17 

15 

-1 

1 

15 

L'ablt:;.a  'jorrecponds  t,o  the 
point  ■ with  w - {“-j/lf,  ,K/1'^  ,f)  > 
sir-;e  y,  io  distinguished, 
un  a temp'  is  r-.ade  to  increase 


blocks. 


After  the  out-pivot,  w^,  though 
nor.basic,  has  the  value  S'V'l'i' 

(at  which  its  increase  was 
blocked  in  the  previous  tableau.' 
Hence  this  tableau  corresponds 
to  the  point  S with 
w = (o,  b/l7,  9/l7).  Continue 
the  increase  of  w_ • 


.‘^ince  > 0 for  i £ I, 
terr.inate  with  tableau 
corresponding  to  the  solution 
T,  having  w = (o,  ll/?6,  15/26'. 


Wolfe's  LDP  algorithm,  applied  to  the  same  problem,  generates  the  following 
sequence  of  points: 


15 
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3.  Analysis  of  the  S-aLgorlthic  applied  to  the  LDP 
a.  Initialization 

This  section  examines  the  structure  of  the  tableau  obtained  by  the 
initialization  step  of  the  S-algorithm.  The  tableau  is  shown  to  correspond 
to  a point  of  P having  minimum  norm.  In  order  to  develop  these  results, 
we  need  to  introduce  the  following  concept. 

Consider  a linear  system  of  tne  foria  w - Mz  + q. 

Definition.  The  tableau  representing  the  system  is  said  to  possess  the 
duplicate  column  property  with  respect  to  the  variable  2^  if  it  can 
be  rearranged  into  the  form: 


2 . z 1 
1 


w 

q 

1 

q 

Lemma  3.0.  Consider  any  tableau  having  the  duplicate  column  property 
■ ii':  respect  to  z^.  Assuiiie  that  some  nonsingular  submatrix  of  M exists, 
•ud  that  a block  pivot  is  performed  on  this  submatrix.  Then  the  resulting 
tableau  also  has  the  duplicate  column  property  with  respect  to  the 
variable  z^. 

Proof : The  result  follows  immediately  upon  partitioning  the  matrix  M 
and  the  vectors  w and  z in  an  appropriate  manner  and  performing  the 
block  pivot.  I 
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V/e  now  examine  the  S-algorithm  as  applied  tc  the  LDP: 


ITT 

minimize  — w p IV 
T 

subject  to  e w = 1 , w ^ 0 . 

The  initial  tableau  is  as  shown  in  Section  2b  of  this  paper.  Recall 

vpin 

that  the  variable  w is  defined  by  w = w.  - 1,  and  that  j.  is 

m-t-i  i-x  1 

T 

the  index  of  the  smallest  element  on  the  diagonal  of  P P-  iThe  S-algoritiim 
resolves  ties  for  the  choice  of  this  index  arbitrarily. ^ The  following 
result  describes  the  effect  of  the  initialization  step. 

Proposition  3.1.  Assume  that  the  S-algorittun  is  applied  to  the  LDP. 

(1)  As  a result  of  the  initial  exchange -pivot 

\i  £ I),  a tableau  is  obtained  which  has  the  duplicate  column 
property  with  respect  to 

(2)  This  tableau  corresponds  to  a point  of  P having  minimum  norm. 

(3)  All  subsequent  tableaux  generated  by  the  S-algorithm  also  possess 

the  duplicate  column  property  with  respect  to  variable, 

having  left,  the  basis  in  the  initialization  step,  never  returns. 


I 

f 


Proof : 'ITie  star*^^nig  tableau,  T*^,  Le: 


m+1 


'^l 

✓s 

V 

^m^l 

1 

..T 

mil 

-1 

0 

n=2 

-e 

0 

1 

T 

e 

0 

-1 

Assume  that  m^^  is  the  smallest  element  on  the  main  diagonal  of 


T 

P P 


m. 


11 


u. 

X 


M. 
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The  initialization  step  calls  for  an  exchange -pivot  •ci’'^l^  ’ ^^l’^m+1^’ 
which  is  of  course  equivalent  to  a block  principal  pivot  on  the  nonsingular 
submatrix 


-1 
k\ 

As  a result  of  this  pivot,  a new  tableau,  T^,  is  obtained.  It  is  easily- 

verified  that  this  tableau  has  the  duplicate  column  property  with  respect 

to  Furthermore,  it  corresponds  to  a primal  feasible  point 

/> 

having  coordinates  w = (w^, w^, . . . ,w^)  = (1,0,..., 0)  with  respect  to  the 
set  P.  By  our  choice  of  it  follows  that  P,  is  a point  of  P 

of  minimum  norm.  Since  is  feasible,  the  initialization  step  of  the 

algorithm  has  been  completed.  (in  the  more  general  case  of  convex 
quadratic  programming,  the  S-algorithm  may  require  not  one  but  several 
exchange -pivots  to  locate  a primal  feasible  point.) 
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starting  from  tableau  T , the  S-algorithm  will  perform  a sequence 

of  principal  pivots.  Lemma  3.0  implies  that  all  subsequent  tableaux 

generated  by  the  S-algorithm  will  have  the  duplicate  coluian  property  with 

respect  to  Of  course,  this  holds  true  only  if  the  variable 

never  becomes  basic  after  the  initialization. 

To  show  that  the  latter  condition  is  satisfied,  observe  that 

any  tableau  generated  by  the  S-algorithm  corresponds  to  a primal  feasible 

point  (i.e.,  w > 0,  and  • Hence  in  each  such  tableau,  we 

must  have  w , , = ,,  w.  - 1 = 0.  By  definition  of  the  vector  y, 

m+1  1=1  1 

T 

®^m+l  = P ^ " y-  Applying  the  conditions  of  primal  feasibility  and 
* 

complementarity, 

m T’  T T T T T 

y = (w  e)y  j^T=wPPw-wy  = wPPw=XX>0 
•'m+l  ^m+1 


We  conclude  that,  in  any  tableau  generated  by  the  S-algorithm,  y^ 
must  be  basic  and  its  complement  must  be  nonbasic. 


Remarks . 


1.  The  initializing  block  pivot  on 


mil 


may  be  interpreted  as  a restriction  of  the  quadratic  fom  to  the 
linear  manifold  determined  by  the  constraints  y^  = 0 and  = 0 

T 

(i.e.  e w = 1).  This  yields  a primal  feasible  basic  solution  with  w^  > 0. 


Note.  A similar  argument  is  employed  for  the  case  of  "noncomplementary" 
tableaux. 


-column  q.re  ibut  for  a sign  change)  precisely 


2.  The  w ^,-row  and  the  y 

m^-l  m*-l 

the  "0-row"  and  "0-colamn"  described  in  Wolfe's  statement  of  his 
algorithm  [11].  Wolfe  points  out  that  "once  having  been  pivoted  in, 
the  [0th  row  and  column]  are  not  used  again  for  pivot  choices." 

3.  In  proceeding  from  to  T^,  the  pivot  sergience  (y^.w^), 

(w  , ,y  could  also  have  been  used,  since  nn , > 0 and  since 

' m+l  m+1  11 

the  appropriate  pivot  element  of  the  intermediate  tableau  will  be 
positive.  This  corresponds  to  the  pivot  sequence  employed  in  Wolfe's 
method. 

b.  Structure  of  the  tableaux  generated  by  the  S-algorithm 

The  main  result  of  this  section  is  a demonstration  that  when  applied 
to  the  LDP,  the  S-algorithm  performs  only  1x1  principal  pivots;  no 
"exchange-pivots"  are  necessary.  This  result  is  based  on  certain  algebraic 
properties  of  tableaux  under  principal  pivot  operations,  some  of  which  are 
reviewed  below. 

Definition.  Given  any  sauare  matrix  M,  the  nullity  of  M is  defined  by 

n(M)  = order  M - rank  M . 

Now  consider  the  homogeneous  linear  system  represented  by  the  tableau: 


yj 

"ii 

1 

21 


■T 


1 

I 

I 

The  following  result,  due  to  Keller,  shows  that  the  nullities  of  the  j 

blocks  and  Mj^.  are  invariant  under  principal  transformations  ! 

(i.e.,  sequences  of  principal  pivots  and  principal  rearrangements'). 

Theorem  3.2  (Keller  p.  22).  Suppose  that  a principal  transformation 

is  performed  on  the  tableau  shown  above.  Let  the  new  tableua  resulting 
from  this  transformation  be  represented  as; 


I'l' 

Ml.,. 

M;,!. 

M 

J*J* 

Then 

n 

= n(M^j) 

and 

ntM.,  ) 

d ' 0 ’ 

II 

uS 

C_, 

Now,  suppose  that  the  S-algorithm  is  applied  to  the  LDP.  Hie 
original  tableau,  T*^,  may  be  written: 


22 


I 


i 


Associated  with  this  initial  tableau  are  index  sets  1^^  - 
and  Jq  = fm*-l}.  Let 


Observe  that  ^ = 0.  rank(M^Q  ) - 0,  and  = 1.  It  follows  that 

J J J 

0 0 T 0 T 

n(M  Q ) = 1.  Similarly,  we  have  M ^ q = P P,  rank.  M ^ ^ = rank  P P = rank  P, 

J II  II 

and  jfl  = m.  Hence  ) = m - rank  P.  These  observations  enable  us  to 

prove  the  following  result. 


Corollary  3-2.1,  Let  Tableau  T be  the  initial  tableau  for  the  LDP. 

Let  T be  any  subsequent  tableau  generated  by  the  S-algoritim,  and  suppose 
it  is  represented  as: 


^I 

yj 

M 

IJ 

^JI 

M 

JJ 

Then  Tableau  T has  the  following  properties: 
(l)  The  matrix 


^JI  ^JJ 


23 


is  blsymraetric,  i.e.  the  blocks  and  M-^,.  are  symmetric,  and 


JJ 


Mjj  = ”^jl’  Furthermore,  14^  is  positive  semidef inite. 
(2>  nullity  (Mjj)  = 1 
(3)  = rank  M + 1 

J J 

(1^'>  det  M__  = 0 
J J 

“jj  is  symmetric  and  positive  semidefinite 

(6)  If  jfj  = 1,  then  M = 0 (scalar). 

Ju 


Proof: 

'ihe  matrix  is  a principal  transform  of  the  original  matrix 

i.e.  it  is  obtained  from  by  a sequence  of  principal  pivots  and  principal 

rearrangements.  Observe  that  is  a bisyrametric  matrix  and  that  it  is 

m 

positive  semidefinite  since  P P is.  By  results  of  Cottle  and  Dantzig  [2] 

and  Tucker  and  Wolfe  (cited  in  Parsons  [6]),  the  properties  of  positive 

semidefiniteness  and  bisymmetry  are  preserved  under  principal  transformations. 

(2)  In  the  initial  tableau  T*^,  we  have  u(M*^q  q)  = 1.  By  Theorem  3-2, 

(3’  By  the  definition  of  nullity  and  the  fact  that  nwM  ) - 1,  we  must 

J J 

have  II J = rank  M 1 . 

J J 

(U ) By  part  (5)>  rank  M , = order  M - 1.  Hence  M 

«J'J  JJ  JJ 

det  I'^j  = 0. 
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is  singular  and 


(5)  This  follows  from  the  fact  that  Mjj  is  a principal  submatrix  of 
the  bisymmetric  and  positive  semidefinite  matrix 

(6)  If  p = 1,  we  must  have  rank  = 0 by  part  (5)  above.  Since 

the  order  of  Mjj  is  1,  it  follows  that  =0.  1 


The  following  i.emma  and  Corollaries  describe  the  structure  of 
the  tableaux  in  greater  detail. 


Lemma  Let 


where  e and  e are  both  column  vectors  of  I's,  though  possibly  of 
different  dimensions.  Suppose 


be  the  principal  transform  of  M obtained  from  a block  pivot  on  the  sub- 
matrix K.  Then 
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I 'roof.  We  have 


(31  e = 1 . 


'fherefore: 

) q'^Q  .^2  - eM^2  = ^ 

(5 ) Q^Q  Mg^  - = 0 

+ 0 ^ 

(7I  e‘  *■  0 M^  = 1. 

E!quations  {6)  and  (7)  are  .just  (?)  and  (5).  To  verify  (l),  we  compute 


E'rom  (2  ' and  (3 ) > we  have 


T T 
(e  Mg^Q^R) 


T,-,  £.T  ^ -T 

e ti  _e  ) = -e 
23 


which  is  (1). 


I 


Corollary  5.3.1.  Let  the  tableau  be  obtained  from  the  initial  tableau  T®  '' 

|i 

k 1 

by  a principal  transformation.  Suppose  that  T is  written  as:  t 


R 

’^mt-1 

ij 

1 

1 i 

f \ 

""r  I 
R R 

rr« 

1 

'■’'k 

M 

I^,m-rl 

^ ^m+l 

Ip,m+1 

M 

m+1  >m+-l 

_..,T 

* J,m-i-l 

M 

mtl,m+-l 

^ \ 

r 

M 

J,Ip 

^J,m+1 

— 

M.. 

j ,m.t  1 

where  the  index  set  is  defined  as  = r,  r+i, 

and  #J  = i = m-r.  Let  e - (l,  1, . . . , 1)"^  £ and  e = (1,1,...,!)'^  t E^. 
Then  the  following  properties  hold; 

(1)  e M,  -r  = -e 

J,  ip 

T T 

(ai  Ajj  - 0^ 

(5> 

Proof:  The  initial  tableau  T^  may  be  written: 


R 

Wt 

tj 

^mtl 

1 

E 

T 

R R 

T 

R Q 

-e 

0 

yj 

T 

Q R 

T 

rQ 

-e 

0 

\+i 

,T 

e 

T 

e 

0 

-1 

To  obtain  tableau  T , a bloct  pivoi  on  i)ie  inatri)' 


muat  be  performed,  llte  required  result  follows  directly  from  Leiruna  3.3  . | 


The  following  result  i.s  a special  case  of  the  preceding  Corollary. 


Corollary  3.3.2.  Suppose  thau  some  tableau  T generated  by  the  S-algorithin 
possesses  the  property  that  ffJ  ~ 1.  Then  the  tableau  must  have  the  follow- 
ing structure: 


I < 


{ 


'^m+1 

yj 

1 

Mr 

lR,m+l 

e 

"Ir,.+1 

M 

m+1,  m+1 

-1 

M 

m+1, m+1 

T 

-e 

1 

0 

1 

this  case 

ffl  = m, 

= m-1,  and  , 

//J  = 1. 

Therefore, 

notation  introduced  previously,  we  must  have  m = r+1,  and  £ = m-r  = 1. 
Hence  the  vector  e is  just  the  scalar  1.  The  result  follows  immediately 
from  Corollary  3.5.1.  | 


Corollary  3.3.3.  In  the  above  situation  (i.e.  when  tableau  T has  the 
property  that  >;J  - li,  if  the  current  solution  X = Pw  is  not  optimal, 
then  the  next  pivot  to  be  executed  by  the  S -algorithm  must  be  either  an  in- 
pivot or  an  exchange-pivot. 
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Proof t Suppose  on  the  contrary  that  an  out-pivot  were  to  be  performed. 


In  the  resulting  tableau,  it  will  hold  that  0=0,  and  hence  w^  = w^ 

=z  . . . = w =0.  However,  we  also  have  w , w.  -1  = 0,  a 

m ’ cH-1  1=1  1 ' 


contradiction. 


Corollary  ^0.h.  Starting  from  a tableau  having  0 > 1,  at  most  0-1 
consecutive  out-pivots  can  be  executed  by  the  S -algorithm. 

Proof t Follows  from  Corollary  3.^.3  and  the  fact  that  each  out-pivot 
decreases  0 by  one.  | 

The  following  observation  on  positive  semidefinite  matrices  is 
found  in  a paper  by  Cottle  [ 1] : 

Lemma  3. If  M is  a symmetric  and  positive  semidefinite  matrix  and  if 
= 0 for  some  k,  then  = m^^  = 0 for  all  i / k. 

The  next  theorem  deals  with  the  case  in  which  the  index  set  J 
has  cardinality  greater  them  1. 

Theorem  3.3.  Let  T be  a tableau  geners.ted  by  the  S-algorithm  in  which 
0 >2.  (Refer  to  the  diagram  in  the  statement  of  Corollary  3.3.1.)  Then 
the  symmetric  positive  semidefinite  submatrix  Mjj  has  positive  elements 
on  its  main  diagonal. 
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Proof;  Recall  that  M 


is  a matrix  of  order  it.  By  Corollary  3.2.1(y' 


JJ 

Mj  , is  s;vT>™etric  and  positive  semidefinite.  By  part  } of  the  sajr.e 

Corollair/'j  rank  M , = order  M - 1 = i-1.  Now  suppose  that  more  than  one 

^ t i O I J J 

of  the  diagonal  elements  of  is  0,  say  = 0.  Using  Lemma 

we  have: 


0 0 0 


M 


JJ 


0 0 0 
0 0 


0 0 m. 


This  matrix  must  have  rank  less  than  or  equal  to  i-2,  which  coni;radicts  the 
fact  that  rank  M , = £-1. 

O J 

The  only  remaining  possibility  is  if  one  of  the  diagonal  elements 


of  Mjj  is  zero,  say  'fhen 


By  hypothesis 
Hence  the  suhmatrix 


H 


are  all  positive. 


Also,  rank  M = f -i > 

J u 
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T T 

has  rank  £-1  and  is  thus  nonsingular.  But  by  Corollary  ^..^>,1,  e M = 0 . 

JJ 

Therefore  e M = 0 v.'here  e - (.1,1,...,!)  . But  this  contradicts  the 
nonsingularity  of  M.  We  conclude  that  > 0,  i.e.  all  of  the  main 

diagonal  elements  of  Mjj  are  positive.  | 

We  now  state  the  rriain  result  of  this  section. 


Theorem  3.6.  After  the  initialization  step,  all  .juccessive  pivots  performed 
by  the  S-algorith-m  are  ixl  principal  pivots.  Thus  no  "exchange-pivots" 
are  necessary  when  the  S-algorithm  is  applied  to  the  LDP. 

Proof:  Let  w,  be  the  driving  variable  in  the  current  tableau.  Either  y^ 

'the  negative  basic  distinguished  variable  which  is  the  complement  of  w^ , 

blocks  the  increase  of  w by  increasing  to  zero,  or  else  some  w 

t s 

is  C.  J ''  blocks  the  increase  of  w,  by  decreasing  to  zero. 

Case  A:  If  y^  is  the  blocking  variable,  then  clearly  m^^^  = 

Hence,  the  in-pivot  (y^,w^  may  be  made. 

Case  B:  if  w is  f d)  is  the  blocking  variable,  and  jlJ  t ?.  llieorem  5.1 

s 

guarantees  that  m t 0.  This  permits  the  out-pivot  'w  ,y  1 to  be  made, 
s s s s 

Consider  finally  the  special  case  where  p}  ' 1.  Hie  tableau  will 
then  have  the  structixre  described  in  Coroilary  and  it  will  correspcnd 

to  the  primal  feasible  point  P . Since  m =0,  an  out-pivot  is  clearly 
impossible  here.  Furthermore,  an  exchange-pivot  {w^,w^’i,  'y^,y^'  is  also 
imnossible,  for  it  would  yield  a new  tableau  corresponding  to  the  primal 


51 


feasible  point  P^,  which  has  a higher  ob]ective  function  value  than  P^. 
(The  fact  that  P^  Is  a.  point  of  minimum  nonn  in  i is  guaranteed  by 
the  initialization  step.)  We  conclude  tiiau  if  «J  u.  i,  the  next  pivot  to 
be  executed  must  be  an  in-pivot,  and  that  must  be  the  blocking 

variable.  | 

Remarks 

1.  We  have  shown  that,  apart  from  the  initialization  step,  no  exchange- 
pivots  in  the  3-algoritlim  a.re  necessary.  However,  by  Remark  t)  followixig 
Proposition  3.1,  the  initialization  can  in  fact  be  a,ccomplishad  by  tw'O 
1x1  principal  pivots  (rather  than  by  an  exchange -pivot ' . Thus,  the 
S-algorithm  can  find  tVie  solution  of  the  LDP  by  using  1x1  principal 
pivots  exclusively. 

2.  Zoutendijk  ([13],  pp.  80-86)  examined  principal  pivoting  methods  for 
solving  linear  complementarity  problems  of  the  form.: 

n 

X' ./  - 0,  X ^ 0,  y > 0. 

These  relations  constitute  the  Kulin-Tucker  conditions  for  the  problem: 

miniiriize  — ||d  - lyii 
sub.ject  to  y > 0 . 

Geometrically,  this  problem^  may  be  stated:  liven  a set  of  points 

P - fPj^.P^. . . . ,P^]  in  and  a point  d t!  find  the  poii;t  closest 

to  d in  the  convex  cone  generated  by  P.  Tn  tiie  conte.xt  of  this  probleiTi, 
Zoutendi.jk  proved  results  which  are  roughly  analogous  to  Theorems  '.5 
and  3-'^  of  tills  paper.  He  also  suggests  a number  of  diffej-ent  algorithmic 
procedures  to  resolve  degeneracy. 


y 1 


T„ 

T 

P P 

-P  d 

3.  An  algorithm  for  finding  the  point  of  a finite  cone  (with  given 
generators)  nearest  to  a given  point  has  been  published  by  Wilhelmsen  [10]. 
In  a footnote,  Wilhelmsen  remarks  that  the  algorithm!  can  easily  be  modified 
to  apply  to  convex  polytopes  and  then  it  becomes  identical  to  that  of 
Wolfe  [11,12]. 


c.  Geometric  j nterpretation  of  the  tableaux  elements 

The  following  propositions  provide  a geometric  interpre;:ation  of 
the  elements  in  the  successive  tableaux  generated  by  the  S-algorithm. 

They  are  reformulations  of  Propositions  1-4  of  Wolfe  [11]. 

Assume  that  the  current  tableau  T is  given  by; 


R 

w 

m+*l 

1 

M 

r I 
R R 

Ip,m+1 

T 

-^r  T 
d,Ip 

^Ip,m+1 

^mnl 

M 

‘m^l,Ip 

M 

ra-t-1  ,m!-l 

J,ra*-1 

M 

Ki*-l,m+l 

w . 

J 

I 

J,  -Lr 

M 

J ,m-i-l 

M 

■ J,mrl 

As  before,  I - Ip  iJ  fm+I]  is  the  current  set.  of  indices  of  the  basic  dual 
variables,  and  J is  the  current  set  of  indices  of  the  basic  primal 
variables . 

M,  , are  the  coordinates 

M ,m!-l 

G = [P  :.1  G J]. 


Proposition  3.7.  The  elements  of  the  vector 
of  the  current  feasible  point  X in  the  set 

33 


Proof:  VJe  have  X - where 


w 

>-  <’ 


In  the  tableau  T correspond ing  lo  X,  however,  w is  nonbasic.  Also, 

^P. 

by  the  duplicate  colum.n  property  of  the  tableau,  tiie  y/'alue  of  'lie  basic 


•variables  v,  is  given  by  M,  Hence 

J d , Tl  ' i 


0 

h 

v;^ 

M 

J 

J ,;nt-l 

> 0 


by  feasibility.  By  Corollary  ^;.3.1, 


m 


E w.  = e M,  , = 1 


i-1 


•ijmi-l 


Remark.  After  each  out -pivot  is  performed,  we  obsaiu  a "non-complementary '' 

tableau  in  the  sense  that  v, y^  > 0 for  some  t I.  Id  such  a 

tableau,  the  values  of  all  primal  basic  variables  w.  (j  - J) 

J 

must  be  modified  to  be  a . •■  m.  ®,  where  is  tl.e  value  at  wiiich 

M ,]t 

driving  variable  w^  (now  nonbasic)  was'  blocked  in  tiie  previous  vableau, 
and  q.  = M.  . '.'iirthennore,  tiie  nonbasic  driving  variable  w Is 
assigned  a positive  value  in  the  "non-complementary"  tableau. 


5'* 


i^oposition  3-6- 


ll)  In  a complementary  tableau,  the  element  gives  the  current 


, ^ fp 

value  of  V , , which  is  equal  to  w'P'^Pw  = X X (where  X = Pw  is  the 
‘^m+'l 


current  primal  feasible  point) 


(2'  In  a non-complementary  tableau,  the  following  identity  holds: 


x"^X  = wVpw  = M , ^ 2M^  - M,  , 

m-t-l,mel  t,rn+-l  tt 


wiiere  ^ is  the  value  at  which  the  increase  of  the  driving  variable  w , 
t € Ij^,  was  blocked;  is  the  component  of  the  vector  ML^, 

corresponding  to  the  variable  y ; and  M is  the  corresponding  diagonal 

Li  tX/ 

entry  of  the  matrix  M-,  _ . 

R^R 


Proof:  These  results  may  be  readily  demonstrated  using  the  approach  of 

van  de  Panne  and  Whinston  [8],  who  ad, loin  a row  representing  the  objective 
fmiction  to  the  Kuhn-Tucker  tableau  and  then  perform  a principal  trans- 
formation on  the  augmented  tableau.  Alternatively,  the  proposition  may 
be  proven  directly,  using  Corollary/  3-3-1-  I 


Proposition  3 •9-  In  a complementary  tableau,  the  value  of  the  elament 

T T 

M.  , (i  6 I„)  is  P.X  - X X,  where  X = Pw  is  the  current  primal 
i,m^l  R 1 

feasible  nolnt. 


Proof : Consider  the  initial  tableau  T shown  in  the  proof  of  Corollary 

3.3-l>  Perform  a block  pivot  on  the  submatrix 


T 

rQ  -e 

T 

e 0 _ 


A straightfoivard  calculation  shown  that  in  tableau  T , 


Finally,  observe  tb.a 


lies  on  ihe  near  sjde  ol‘ 


then 


91a  ' 


pTx  < X^X.  [n  such  a cnse  M. 


.-T„ 


i ,inf-l 


P.'X  - X X'  < 0,  and  so 
a 


"i.ri ' -"’'“■a* 


I'lii 


Proposition  3*10.  Q = {P-^j  € ^ }*  Define  the  point  W.  to  be  the 

J ^ 

m 

projection  of  the  point  P^.  onto  the  affine  hull  AvQ,)  = fZ:Z  = Qu,  e*u --  1], 
where  i G Ip  = l\  (n+l].  then: 

T 

(l)  For  i £ 1„>  the  ith  rovf  of  the  matrix  -M,  gives  the  coordinates 

P 

u of  the  point  Xh  with  respect  to  the  set  of  points  0.  ir.  other  words, 


[ -M  ] . 

J,Ip  1. 


and  = Qu  . 


(2)  The  entry  M. . (i  £ on  the  main  diagonal  of  M,  _ equals 

the  square  of  the  distance  of  P.  from.  A(q). 


Proof: 

(1'  The  problem  of  projecting  onto  A,Q'  may  be  fomiulated  as: 

2 T 

minimize  ||p^  - Qu||'^,  subject  to  e u = 1.  As  shown  by  V.'olfe  [IC], 
necessary  and  sufficient  conditions  for  a solution  "c  this  problem  are: 


T T 

Q Qu  - e>  - Q,*P. 

e'u  - 1 

Fmploying  the  notation  of  lemma  3-5  and  Gorollar?/  3-5-1-  we  have 


u 

rp 

Q^Q 

-e 

-1 

q"p. 

■%5 

Q‘P. 

1 

'A 

T 

e 

0 

1 

S5  . 

1 

57 


ioweV'T,  i*:  cm  also  be 


Hence  u - '•  ^ 'V , 

shown  that 


-M 


’ T - 

: j ■ = R Q>^p 


eM, 


25 


T 

R 0>1 


J.3 


eM^' 


mt-1 


The  ith  row  of  this  matrix  is  clearly  identical  to  u . Hov^  se'*ing 
Qu  yields  the  projection  i.f  on^o  A.(q). 


(2)  Using  Lemma  .3  and  Corollary  3-5 -1)  we  obtain: 


R R 


Tlie  (i,i)  diagonal  elem.ent  of  this  matrix  reduces  t: 


M . = pTp  - p'^'qu  - P^’^Ou  + V/%J.  = (,F.  - V.\'-^’>P.  - W.  ) 

'll  11  1 i ' 11  1 1 1 1 


I 


U . Equivalence  of  the  a 1^ o ri thms 

To  show  that  the  S-algorithm  and  Wolfe's  I.uP  algorithm  generate 
the  same  feasible  points,  we  need  to  demonstrate  that  the  corresponding 
steps  of  the  two  methods  achieve  identical  results. 

Assume  that,  at  some  Iteration  of  the  S-algorithm,  a basic 
complementary  tableau  T has  been  obtained,  corresponding  to  a primal 
feasible  ^but  not  optimal)  point  X^.  Such  a poitU  is  denoted  by  Wolfe 
as  a point  of  minimum  norm  in  the  convex  hull  C\Q  ' of  a cer'ain  corral 

3P 


Suppose  that  the  tahle^u  h;  s ttse  f..>l].ovin^  ;-tru;'ure 


denote  the  constant  column 


corresponding  to  nhis  tableau  has 


barj’-centric  coordinates  w. 


points  P.  In  V/olfe's  ter 


.he  current  corral  0 


of  the  uoints  P and  'P 


Without  loss  of  generality,  we  may  a.ssuine  that  the  most  negative 


< 0.  Hence  the 


lO  S u: 


current  noint  a' 


By  Proposition 


2.1  of  Wolf 


X is  the  .solution  of  the  LDP 


f and  only  if  X F.  > X‘X 


step  1 of  Wolfe’.s  method  determine 


whether  this  condition  Is  satisfied 


'Fhus , in  this  case  tlie  current 


-X  would  be  found  non 


optimal  by  Wolfe's  method 


- 

S u 

■P 

T 

-M 

-K 

si 

, m »■  I 

-?/  ’ 

T 

T.-, 

V 

r/^ 

-Is 

jj 

r 

In  this  situation,  an  attempt  is  made  in  the  S-algorithm  to 
increase  (the  complement  of  the  distinguished  variable  y^).  By 

Lemma  3-4,  it  follows  that  > 0 (for  if  = 0,  then  = 0, 

contradicting  the  assumption  that  < O’*. 

A preliminary  ratio  test  is  then  performed  to  find  an  index  s 
such  that 

f 

I. 

[ 

F. 
r 

[ In  the  notation  of  the  above  tableau,  we  have 


s,m-t-l  j ,m+l 

M ^ - M.. 

St  jt 


for  all  j £ J for  which  M.  < 0. 

Jt 


Since  M , = w > 0 by  the  nondegeneracy  assumption,  it  follows  that 
s ,m+l  s 


M 


s ,m+l 


M 


< 0 


st 


The  next  step  of  the  S-algorithm  is  to  determine  whether  the  basic 

variable  first  blocking  the  increase  of  w^  (i.e.  attaining  the  value  of 

zero)  is  y,  or  w . To  accomplish  this,  two  ratios  are  compared,  namely 
t s 


See  Appendix  I. 


M 


t,m+l 


M 


< 0 and 


tt 


M 


s ,m^-l 


M 


< 0 


st 


4o 


There  are  two  cases  to  consider: 


1 


Case  A.  Suppose  that 

T/  ;,f 

M C-  i7~  ' ■ 
tt  "St 

Then  blocks  the  increase  of  before  i,or  simultaneously  as) 

w decreases  to  0.  I'Jhe  S-algoriiLm  always  resolves  blocking  ties  in 
s 

favor  of  the  distinguished  variable  • ) In  this  situai.ion,  the  in-pivot 

K ^'1 

is  executed  to  yield  a rev  tableau  T corresponding  to  a 
k+1 

point  X which  is  primal  feasible.  As  will  be  shown,  that  is  precisely 
the  same  pivot  as  is  performed  in  Step  2 of  the  tableau  vari.ant  of 
Wolfe's  LDP  algorithm. 

k tl 

In  the  tableau  T resulting  from  the  in-plvot,  the  basic 
primal  variables  are  w^ , w^  and  w , and  their  values  are  given  by: 


w,. 

U 

W 

£ 

w 

.J 


In  V/olfe ' s t 

k^l 

erminology,  the  new  corral  Q is 

co:nposed  of  the 

poiiitE 

P.  , P , and 
t s 

{P^:,1  c-  . The  new  feasible  poiiV 

k- 1 

X ;.ac  bair.n 

.^entri  c 

coordinates 

w^ , w , and  w with  rcrpec*  '<  'hi 

t G T 

Sc'  o:'  p -in'  s 

-K^i 

•V 

i'i 


To  demonstrate  the  fact  that  the  two  algorithms  generate  the 


same  point,  it  must  be  shown  that  X 


kt-l 


is  identical  to  Y,  the  point 


of  smallest  norm  in  and  furthermore  that  X £ rel  int 

Since  the  ratio  test  indicated  that  an  in-pivot  was  possible, 

k+1  T 

the  point  X = Pw  must  be  primal  feasible.  Hence  w > 0 and  e w = 1; 

in  particular,  w,  + w -t-  S - w.  = 1.  Assuming  nondegeneracy,  all  of  the 
t s J J J 

barycentric  coordinates  of  are  positive.  Hence  € rel  int 

k+1 

Now,  let  Y be  the  point  of  smallest  norm  in  A(Q  ).  Y can  be 
determined  by  solving  the  problem: 


minimize 


llOhir 


T _ k+1 

subject  to  e u = 1,  where  Q = Q 


Substituting  the  point  0 for  the  point  in  the  proof  of  Proposition  5.10, 

we  find  that  the  barycentric  coordinates  of  Y with  respect  to  the  set 

-k+1  . , 

Q are  given  by: 


T 

M ftp  + 
JJ^  i 


M 


. = M 

J,m^-1  J,m+1 


But  the  vector  , represents  the  current  values  of  the  basic  variables 

<J  ) J. 


w 


w 

U j j 


k+1 

in  Tableau  T , since  J = (t]  U {s]  U J.  In  other  words,  Mj  is 

k+1 

the  vector  of  barycentric  coordinates  of  the  point  X with  respect 

k^l 

to  the  set  of  points  Q 


Since  is  nn  affim-ly  Independen'.  sei,  any  poir.',  in 

V rl 

will  have  a uniq;re  bar^/'centric  represet:T,ation  recpect,  io  Q' 

k.i-1 

decause  X and  Y have  ihe  same  baiycentr^c  coordinates,  n-xmely 

kel 

M,  , - u,  thev  must  be  identica]  . Hence  the  t)oin+.  X obtained 

djir.*-! 

after  an  in-pivot  of  the  S-algorithm  is  indeed  the  point  of  minimum  norm 

-k*'l\  k^l  kyl 

in  A(.Q  and  furthermore,  X € rel  int  CiQ 


Case  B.  Suppose  the  ratio  test  indicates  the 


V,  M 

‘ t,tr.rl  s ,m+l 

4.  ^ W 4- 

tt  St 


Then,  w blocks  the  increase  of  w,  (i,e,  decreases  to  0!  before 

S w 

y increases  to  0.  In  this  case,  the  S-aigorithm  performs 

b 

\c 

the  out-pivot  (w„  •y^'i  in  the  tableau  T . As  a result,  a new  tableau 
s s 

is  obtained  which  we  denote  as  T vto  avoid  confusion  v^tth  T 
considered  in  Case  A).  For  the  purpose  of  the  proof,  ‘he  relevant  portion 


of  the  tableau  T is  given  by; 


V 

St 

M - “ M 


Tableau  T is  "norcompleraentary"  in  the  pcr.se  that  the  dri\d.ng  variable 
w^ , although  nonbasic,  has  a positive  value  tlie  value  at  wtiich  its 
increase  was  blocked  by  w^-  "he  value  of  ^ is  determined  by  the  ratio 
test  to  be: 


I. 


Since 


M 


& = 


s,m+l 


M 


st 


Since  has  the  value  6 in  tableau  T , the  values  of  the  basic 

primal  variables  w must  be  adjusted  accordingly; 

j 


w = 


M 


st 


M 


Jt  ss  Js 


M 


6 + M 


J,  m+1 


s,m+L 

M 


M 

js 


M + M 


M 


st 


Jt  j,mtl 


if  • "if 

Let  Q = {P.  ] U (P.:j  c J},  and  let  X be  the  primal  feasible  point 
t 0 

corresponding  to  the  new  tableau  T . Then  the  barycentric  coordinates 

* * 
of  X with  respect  to  the  set  Q are  given  by 


w 

L J 


where  w has  the  value  5 > 0 specified  above.  Since  X is  feasible, 
we  must  have 


> 0 


and  w,  + Z.^7  w.  = 1. 
t J 


UU 


To  sumiriar:  ze  the  notation  uoe^l  here,  we  have; 

k “* 

Q = {P  ] U { P . ; i = originel  corral  correruondi  ng  to  tableau  T 

s J 

k 

X = origina.1  feasible  point  corresx'onilng  to  T‘ 

= {P*)  U fP  i Li  - ji 

t '3  J 

k*-! 

= corral  obtained  by  adjoining  the  point  to  Q by  means 

k 

of  an  i.n-pivot  in  T . 

k i 1 

Y = point  of  sinallest  norm  in  A(Q  Ac  sho’wn  in  case  A,  this 

k 

point  TCuld  have  been  generated  by  an  in-pivot  in  T had 
the  ratio  test  permitted  such  a j'livct. 

0*  = {P^}  U (P^:j  ■;  j) 

* / \ k 

X = point  generated  by  an  out-pivot  in  T 


^incHy,  define  Z to  be  the  point  nearest  to  Y on  the  line  segment 

C(Q*^^^'  n X^.  The  point  Z is  calculated  in  Ptep  x of  V/oife's  LDP 

method.  To  demonstrate  that  the  S-algorit'nm  arrives  at  the  same  point, 

<- 


it  suffices  to  show  that  Z 
proved; 

(1'.  x’'  e 


(2,  x'"  ^ X^ 


X . Thus,  there  are  three  facts  to  be 


)(r 

Any  movement  from  X toward  Y along  the  line  segment 
will  result  in  a pioint  which  is  exterior  to 


X and  Y,  we  must  show  that  0 < A < 1 in  the  equation; 


imbination  of 


X*  = (l-A)X^  + AY  . 

This  equation  can  be  rewritten  in  terms  of  the  barycentric  coordinates 
* k 

of  the  points  X , X ^ and  Y with  respect  to  the  entire  set 
P = {P  , P , P , P } Using  facts  established  earlier  in  this  discussion 

f ^ ^ j ’ 

we  have: 


Solving  this  vector  equation  for  A,  we  obtain 


U6 


A is  positive  since  M,  , > 0,  M,  0,  M , > 0 and  M ^ < 0 

tt  t,m+l  s,rr.+  I st 

by  hypothesis.  Furthermore,  the  outcome  of  the  ratio  test  in>licates  that 


M y 

t.m+1  ,,  ‘s.m+l 

M M 

to  st 


hence  ^ m+1’  implies  tfiat  \ < 1.  V/e  conclude 

that  X = (i-A'X  + AY  with  0 < >.  < L,  i.e.  X can  be  expressed  as 

. * * , * 
a convex  combination  of  X and  Y.  Also,  since  X e CfQ  and 

k+ 1 k 4- 1 

Q c Q , it  fellows  that  X e C(Q  ). 

It  remains  to  verify  fact  (3'.  In  the  vector  equation  above, 

note  that  the  value  of  the  w^-coordinate  a?  a function  of  A is  given  by: 


M . -M 

w = (1-X'  M . , + XM  , T - X — ~ = M - >, 

s s,m+l  s,ra+l  s,m+l 


T -M 
t,m+I  st 

M,  . 
tt 


If  A = A,  we  have  = 0.  For  feasibility  (i.e,  X^  -I  we 

require  that  w >0.  Setting 


K • M 

M - A > 0 

s,m+l  — 

VO 


implies  that  A < A.  l/i  other  words,  any  increase  in  A beyond  the 
value  A will  result  in  an  infeasible  point. 

To  complete  the  eauivalence  proof,  it  must  be  shown  that  the 
critical  value  A is  in  fact  identical  to  the  value  P determined 
in  Step  3 of  Wolfe'  s LDP  algorithm.  This  step  calculates  the  ratio: 


= min 


w . 


— : w . -y  . >0 
w,-,y. 


ki 


■> 


where  w denotes  the  barycentric  coordinates  of  X with  respect  to 
P,  and  y denotes  the  barycentric  coordinates  of  Y with  respect  to 
P.  But  these  coordinates  are  just; 


0 

0 

0 

M 

t,m+l 

M 

tt 

w = 

^^s , m+1 

and  y = 

M *M 

t,m+l  st 

s,m+l 

M 

J,  m+1 

M.  -M 

t,m+l 

M " M 

J,m+1  tt 

Thus, 


9 = min< 


M 


s , m+1 ^ 


M 


t,m+l  st 


M 


tt 


, min 


M. 


■j,m+l\  M. 


M.  ^ -M.. 
t,m+l  ,Tt 


-1 


:j  e J 


tt 


By  the  ratio  test, 


s,nd-l  ^ .1 , m+1 

M ^ - M.. 

st  jt 


for  all  j £ J for  which  M.,  < 0 


Since  M.  , <0,  it  follows  that 
tt  t,m+i 


M 


/m  -M  /m  m 

/ t^m+l__st  \ < f t,m+l  Jt  J 

J - A ^tt 


Hence 


e = M 


N1  ^ *M  , 
t,m+l  st 


-1 


s,m+ll  M. 


tt 


1 


U8 


which  is  identical  to  the  critical  value  a calculated  earlier-.  We 


conclude  that  in  Case  B,  the  S-algoi-ithm  and  V.'olfe's  LD?'  algor^tlun 
generate  identical  primal  feasible  points.  | 

Remark.  The  only  diffe?'erice  between  the  t^o  talgcrithms  lie.-  in  the 
fact  that  the  S-algorithm  first  perform.:  a feasibility  test  ('namely,  the 
ratio  testl  to  aetermine  whether  an  in-pivot  or  an  out-pivot  should  be 
executed.  The  for-mer  case  leads  to  the  start  of  a neu  major  cycle;  the 
latter  ca.se  corresponds  to  a minor  cycle.  Note  that  all  tableau:< 
generated  by  the  .algorithm  correspond  to  primal  feasible  points. 

The  tableau  variant  of  Wolfe’s  LfP  algorithm,  on  the  other  hand, 

<>.•  ..  .*  ■■•■rform'  the  in-pivot,  then  performs  a.  test  to  deterriine  whether 

t:.e  -i’lt  Y corresponding  to  the  nevr  tableau  is  feasible  (i.e.,  lies 
krl. 

'nt  C(Q'  '• ) . If  so.  the  algorithm  begins  a new  major  cycle. 

. it  is  possible  for  the  in-pivot  to  generate  a tableau  corres- 
....  to  a point  Y -.•.’1.1  ch  is  infeasible.  In  this  case,  it  is  then 
necessary  to  e.nter  a minor  cv'cle  (step  (ji ' , which  involves  calculating 
-^tio  d an.i  setting  the  new  point  equal  to  + PY.  As 

has  been  sho-o-n,  this  minor  cycle  achieves  t.he  ssjrie  result  as  does  the 
out-pivot  step  of  the  3-algorithm.  Thus,  although  the  two  methods 
gene,  ate  different  tableaux  in  thei r respocujve  minor  cycles,  the 
sequence.;  of  primal  feasible  points  ieter.mined  the  algorithj.is 
are  identical.  This  point  is  illustrated  by  applyir.g  the  two  algoritlims 
to  Wolfe’s  example: 


Example. 

Assume  that  we  start  from  the  feasible  point  R having  barycentric 
coordinates  (9/l5,  ^/13,  0).  The  corresponding  tableau  is: 


^3 

W4 

^1 

^2 

1 

^3 

1+9 

-42 

17 

-4 

-42 

1 

-42 

36 

-9 

-4 

36 

^1 

" 13 

-17 

9 

1 

-1 

9 

^2 

4 

4 

-1 

1 

4 

r 


corresponds  to  the  point  H 
with  bar-ycentric  coordlnntes 
(O,  3/l6,  9/17).  Continue  the 
increase  of  w.,.  Ratio  test 
shows  that  bl.ocks . 


OaLcuJate  0 - Computing 

'1-3'R  yielif  t ie  point  S. 

Since  e is  attained  for  the  index 
i = 1,  perform  the  pi>/ot 


-Pr/OT 

\y-. 

’ V 

Wi 

yp 

1 

49 

21 

9 

17 

21 

yt 

1 

21 

9 

-11 

-15 

9 

^2 

26 

-9 

11 

1 

-1 

11 

”^5 

-17 

15 

11 

1 

51 


The  preceding  results  may  be  summarized  as  follows: 


Theorem  i. 1. 

(l'  When  applied  to  the  LDP,  the  S-algorithm  of  van  de  Panne  and  Whinston 
and  Wolfe's  LDP  algorithm  are  equivalent  in  the  sense  that  they 
generate  the  same  sequence  of  primal  feasible  points. 

(2'i  Furthermore,  the  successive  taoleaux  generated  by  the  two  methods 
are  identical,  except  during  certain  iterations  in  which  Wolfe's 
algorithm  produces  a tableau  corresponding  to  an  infeasible  point. 

In  such  cases  Wolfe's  method  performs  a minor  cycle  to  restore 
feasibility;  this  step  achieves  the  same  result  as  does  the  corres- 
pondirig  minor  cycle  in  the  S-algorithm. 

(3)  Both  the  S-algorithm  and  the  tableau  variant  of  Wolfe's  method  rely 
exclusively  on  1x1  principal  pivots  in  solving  the  LDP.  The  same 
principal  pivots  are  executed  in  both  methods,  although  possibly  in 
different  order. 

Remarks . 

1.  van  de  Panne  and  Whinston  [8]  showed  that  in  the  case  of  convex  quadratic 
programming,  the  S-algorithm  generates  the  same  sequence  of  primal  feasible 
points  as  does  the  "asymmetric"  algorithm  due  to  Dantzig  [3].  Since  the  LDP 
is  a special  case  of  the  convex  quadratic  programming  problem,  Wolfe's  LDP 
algorithm  is  also  equivalent  fin  the  sense  described  above'  to  Dantzig' s 
method  applied  to  such  a problem. 


r 


I 

I 

1 

I 

j 

i 


i 

I 

j 

I 

I 

i 


1 

1 

",  From  the  computaticnal  point  of  view,  the  S-aiporitliri!  and  the  table  u i 

variant  of  Wolfe's  algorithm  are  also  equival.ent.  The  "duplicate  column"  | 

property  and  the  bisymraetry  of  the  tableaux  generated  by  the  S-method  i 

O i 

imply  that  (m+l)^'/^  storage  locations  are  required.  | 

It  might  appear  that  performing  a minor  cycle  (step  (3))  in  ] 

Wolfe's  method  (which  involves  calculating  a parameter  0 and  determining 
the  new  feasible  point  as  a convex  combination  of  and  Y entails  | 

additional  computational  effort  beyond,  that  required  in  the  S-algorithm.  I 

This  is  not  the  case,  however,  since  the  latter  algorithm  performs  the  j 

equivalent  computation  of  the  ratio  test  and  the  ad.justment  in  the  values  I 

of  the  basic  variables  after  an  out-pivot  has  taiten  place.  | 
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Appendix  I;  On  degeneracy 

In  order  to  guarantee  that  the  S-algorithm  will  solve  the  LDP 
in  a finite  number  of  pivot  steps,  it  is  not  necessary  to  make  the  non- 
degeneracy assumption  (i,e.  the  assumption  that  in  every  tableau,  the 
basic  primal  variables  are  all  strictly  positive). 

To  see  this,  suppose  that  the  current  tableau  T has  the 
structure  depicted  in  the  first  diagram  of  Section  4.  Assume  that  the 
current  complementary  solution  is  not  optimal,  and  that  y^  is  the  most 
negative  basic  dual  variable  (so  that  M < 0).  Now,  contrary  to 

w y ni'r  X 

the  nondegeneracy  assumption,  suppose  that  = 0.  (The  other 

basic  primal  variables  comprising  the  vector  M may  be  taken  to 

J,m+1 

be  positive.) 

The  ratio  test  of  the  S-algorithm  will  determine  that 


M M 

t , m+1  ^ s,n^l 


M. 


M 


0 . 


‘tt  ‘ st 

Hence  "Case  B"  obtains  and  an  out-pivot  must  be  executed  on  the  element 
M 


ss 


(M  is  positive  by  Theorem  3.5.'  The  resulting  tableau  T , as 

SS 


indicated  in  the  discuss'‘on  of  the  preceding  section,  will  have; 


M 


w_  = M_ 

J j, m+1 


s,m+l  ,,  „ 

i M = M 


M 


- 0 > 0 . 


ss 


Js  j,m+l 


Hence,  a zero-valued  basic  barycentric  coordinate  (basic  primal  variable^ 
in  any  tableau  may  be  "removed"  by  making  an  out-pivot. 


Appendix  lit  von  Hohenbalken' s method 


In  this  section,  an  algorithm  due  to  von  Hohenbalken  [9]  for 
maxiini/ing  certain  pseudoconcave  functions  on  polytopes  is  considered. 

It  will  be  shown  that  this  algorithm,  when  specialized  to  the  LDP,  is 
identical  to  Wolfe's  method. 

In  the  following  description  of  von  Hohenbalken' s method,  the 

notation  employed  in  section  2 of  this  paper  will  be  used.  In  addition, 

certain  maximization  problems  will  be  converted  to  equivalent  minimization 

problems  for  the  sake  of  clarity. 

Following  the  initial  step,  the  algorithm  continues  with  a 

sequence  of  major  cycles,  each  of  which  begins  at  Basic  Step  1.  Let 

Q be  the  affinely  independent  subset  of  P at  major  cycle  k,  and 

let  be  the  minimizer  of  f(X)  on  C(Q^),  where  X^  belongs  to 

k k 

the  relative  interior  of  C(Q  ).  (Thus,  Q,  is  a "corral"  in  Wolfe's 
terminology. ) 

j 


VON  HOHENBALKEN’S  ALGORITHM 


SPECIALIZATION  TO  THE  LDP 


Initial  Sten 


Find  an  extreme  point  of  the 


In  the  case  of  the  LDP,  without 


feasible  region.  In  general,  this 


loss  of  generality  we  may  take  this 


is  accomplished  hy  solving  a linear 


extreme  point  to  be  one  having 


program.  Denote  this  extreme  point 
k+1  X 

as  X = X (the  superscript 
refers  to  the  cycle  number) . Go  to 


minimum  norm;  it  may  be  denoted  by 
P?.  With  this  choice,  the  same 
starting  point  is  selected  as  in 


basic  step  1. 


Wolfe's  LDP  method.  Set  X = PV 


Go  to  basic  step  1. 


VON  HOHENBALKEN'S  ALGORITHM 


SPECIALIZATION  TO  THE  LDP 


the  step  tfiAen  in  Step  1 of  Wolfe's 
method  (see  Wolfe  [11,  pages  10  and 
23]). 


In  the  case  of  the  LDP,  X Is 

just  the  extreme  point  P . which 
m J 

minimizes  X P..  Hence  the  ex- 
1 

pression  in  Basic  Step  l(a)  is 


T T 

equivalent  to  8X  P . - 2X  X =0. 

t] 

The  algorithm  terminates,  therefore, 


T T 

if  X^  X^  = X^  P..  In  this  case 
J 

X is  the  optimal  solution  of  the 

LDP.  Note  that  Wolfe's  method 

employs  the  same  optimality  test; 

Theorem  2-1  (Wolfe  [11])  states  that 

X is  optimal  for  the  LDP  if 

x'^P  . > x'^X  for  all  j . 

J - 

If  7f(X^)'^[X-X^]  <0  (i.e. 

T T 

if  X^  P <X^X^),then  X^  is 
clearly  not  optimal  and  the  method 
continues. 


1 

5 

j 


! 
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VON  HOHENBALKEN' S ALGORITHM 


SPPOIALIZATION  TO  THE  LDP 


Basic  Step  2 

There  are  two  cases: 
(a)  If  f(X)  < f(X^^  set 
X^^^  = X, 

= (X]  , 

and  go  to  Basic  step  1. 


(bl  If  f(X)  > f(X^), 

k ^ 

augument  Q by  X 

to  form  a new  affinely 

independent  subset  Q. 

Go  to  basic  step  3. 


When  the  method  is  applied  to 

the  LDP,  case  (a)  will  never  occur. 

Recall  that  the  starting  point  P/^ 

i 


was  chosen  to  have  minimum  norm, 

i.e.  pj  P/v  < P7P.  for  all  i. 
.—  11 

Hence.  X = P.  and  X-^  = P^, 

^ i 


we  must  have: 


m m 1 T 1 1 

f(x)  = P.P.  > pip^  = X X = f(x  ) 

a d - i i 

So  case  (b)  clearly  occurs  here. 
Now,  let  X^  be  the  feasible  point 
available  at  the  beginning  of  major 
cycle  k.  It  is  clear  that 


ffX^)  = X^  X^  < pjp^  < pTp.,  all  i 
- . . — 11 
1 1 

/N 

Since  X must  be  one  of  the  points 
Pj^,  it  follows  that  f(X)>f(X^), 
i.e.  case  (b)  occurs  here  as  well. 
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VON  HOHENBALKEN’ S AL(JORITHM 


SPECIALIZATION  TO  THE  LDP 


Basic  Step  .5 


Attempt  to  find  a lainimizer 


This  is  identical  to  step  2 of 


Y of  f(X)  on  the  linear  manifold  Wolfe's  LDP  algorithm. 
A(Q'. 


(a*!  If  such  a point  Y exists 


Alternative  (b  will  never 


it  satisfies  f(Y)  < f(X  V 


arise  in  the  case  of  the  LDP 


Go  to  basic  step 


because  fCX}  = X*X  will  always 


(b',  If  f does  not  have  a 

minimize  on  A(Q^,  find  its 


have  a minimi zer  on  the  manifold 


A(Q1. 


minimizer  Y'  on  A',  where 


A'  is  the  linear  manifold 


through  X and  parallel  to 


A(Qi 


Go  to  basic  step  5. 


Basic  Step 


The  barycentric  repre- 


In  the  case  of  the  LDP,  we  have 


sentation  of  Y on  A(Q‘!  is 
Y = Qw*,  where  the  columns  of  Q 
are  certain  extreme  points  of  the 
feasible  region. 


Q = {P^,P2, . . and  thus 

-X-  ^ 

Y = Qw  - PtWt+  P„w  + •••  + P.W.. 

112  2 0 0 
* 

Note  that  w . > 0 . 

0 


There  are  two  cases; 


VON  HOHENBALKEN'S  ALGORITHM 

SPECIALIZATION  TO  THE  LDP 

(a)  If  > 0 for  all  i = 1, . . . , j, 

This  is  identical  to  step  2(C) 

then  Y belongs  to  the  relative 

of  Wolfe’s  LDP  algorithm  (the 

interior  of  C(Q)  and  minimizes 
f on  C(Q'i. 

Set  = Y, 

feasibility  test). 

- Q 

and  go  to  basic  step  1. 

(b^  If  < 0 for  some  i,  i f 

then  Y ^ rel  int  C(Q). 

Go  to  basic  step  5* 

Basic  Steo  5 

* 

Intersect  the  boundary  of 

In  the  case  of  the  LDP,  we 

C(Q)  with  the  line  segment  X^. 

may  write  Z - P-j_w^  + 

Let  the  intersection  point  be 

Z = Q,w. 

The  point  Z satisfies 

This  step  is  identical  to 

f(Z)  < f(X^),  and  will  have 

step  3 of  Wolfe's  LDP  algorithm, 

w.  > 0,  all  i,  with  w.  > 0 and 

provided  the  stipulation  is  made 

at  least  one  w^  = 0 for  i j. 

in  von  Hohenbalken' s method  that 

Drop  the  vertices  from 

only  one  point  may  be  dropped  from 

Q that  have  w^  =0,  to  get 

0,  during  basic  step  5- 

a reduced  affinely  independent 

set  Q. 

6l 


VON  HOHENBALKEN’S  ALGORITHI4 


SPECIALIZATION  TO  THE  LDP 


There  are  two  cases: 

(a)  If  C(Q)  is  zero-dimensional 

Assume  that  case  (a)  occurs. 

(i.e.  contains  only  a single 

i.e.  C(Qi  consists  only  of  the  ; 

k-t-1 

li 

point i , set  X = Z, 

point  Z.  von  Hohenbaiken' s method  i| 

= Q, 

will  return  to  basic  step  1 and 

j 

and  go  to  basic  step  1. 

test  point  Z for  optimality.  1 

Wolfe's  method,  on  the  other  hand,  j 

^ I 

(b"!  If  C(Q'i  has  positive 

i 

will  first  determine  that  Z is 

1 

dimension,  set 

the  point  of  smallest  norm  in  ' 

= Z, 

.rw  .1—1  ' 

A(Q)  (since  0 is  a singleton 

Q - Q, 

this  is  the  only  possibility ‘ , 

go  to  basic  step  3. 

and  then  return  to  step  1 for  the 

optimality  test. 

It  is  clear  that  the  steps 

taken  by  the  two  methods  are 

identical.  The  same  holds  true  if 

case  '\b)  occurs. 

The  preceding  observ^ations  may  be  summarized  as  follows: 

Proposition  ".1.  Suppose  that  von  Hohenbaiken' s algorithm  is  applied  to 
the  LL'°,  with  the  following  stipulations: 

(l)  The  starting  point  is  chosen  to  be  a point  of  P of  minimal  norm,  and 
( ■'  In  basic  step  o,  only  one  point  at  a time  may  be  deleted  from  the  set  Q. 


(2 


Then  every  step  taken  by  the  algoritlun  is  identical  to  the  corresponding 
step  of  Wolfe's  LDP  method. 


i; 


Remark.  It  follows  that  von  Hohenbalken' s algorithm  generates  the  same 
sequence  of  primal  feasible  points  as  does  the  van  de  Panne-Whinston 
method  applied  to  the  LDP.  The  extent  to  which  this  observation  may  be 
generalized  to  quadratic  programs  of  a more  general  nature  is  under 
investigation. 
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